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Writing Assessment 
Framework 




Overview 



T his section contains the Writing Framework for the 2007 Na- 
tional Assessment of Educational Progress (NAEP) as adopted 
hy the National Assessment Governing Board (Governing 
Board). The Framework descrihes the basis for the Writing Assess- 
ment, the types of writing assessed, and the methods for scoring stu- 
dent responses. This Framework was developed during 1989-90 in 
preparation for the 1992 NAEP Writing Assessment. For the 1998- 
2007 assessment the Framework was augmented hy a set of Writing 
Assessment and Exercise Specifications developed during 1995-96 
(see pages 23 to 67). 

This document contains the framework and specifications for the 
NAEP 2007 writing assessment, which will assess student achieve- 
ment nationally in grades 8 and 12, continuing the trend begun in 
1998. Additionally, data for participating states and select urban dis- 
tricts will be gathered for grade 8. 

Developed by a committee of writing researchers, teachers, curricu- 
lum specialists, and business representatives, the Writing Framework 
builds on two decades of NAEP experience in large-scale direct writ- 
ing assessment. In addition, the assessment incorporates important 
changes that reflect findings and recommendations from recent re- 
search on writing instruction and assessment, as well as the experi- 
ence of many state writing programs. In developing this Framework, 
input was received from a wide array of writing educators, policy- 
makers, scholars, and major education organizations. Highlights of 
the NAEP Writing Assessment include: 

• Assessment of narrative, informative, and persuasive writing. 

• A set of writing topics that incorporate a variety of stimulus mate- 
rials, audiences, and forms of writing. 

• A special page accompanying each topic for students to plan and 
organize their writing. 

• Enhanced 6-point scoring criteria. 
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Introduction 



T he fundamental aim of writing is to communicate. However, 
its purpose, audience, form, and subject matter vary according 
to the specific writing situation. Good writers can communi- 
cate well in a range of situations. They can perform a variety of writ- 
ing tasks, ranging from business letters to stories, reports, and essays. 
To become good writers students need expert instruction, frequent 
practice, and constructive feedback. 

Although some testing programs have used only multiple-choice 
exercises to assess writing, NAEP has been a pioneer in collecting 
actual samples of student writing and scoring them in a consistent 
way. In recent years, use of direct writing tests has increased. Cur- 
rently, more than 35 states use direct measures of writing in their stu- 
dent assessment programs. Writing samples also are collected as part 
of the SAT, (ACT), Medical College Admissions Test, and a number 
of other large-scale assessment programs. 

In addition to measuring writing, NAEP continues to provide the 
only nationally representative data on student achievement in reading, 
mathematics, science, U.S. history, geography, and other academic 
subjects. Since its inception in 1969, NAEP has assessed representa- 
tive samples of students aged 9, 13, and 17. In 1984, it also began 
sampling students by grade level and currently provides data for 
grades 4, 8, and 12. NAEP reports achievement results by 
race/ethnicity, gender, and region, as well as data that chart trends in 
achievement across time. Relationships are reported between student 
achievement and relevant background factors such as instructional 
practices, courses taken, and homework. 

As part of the 1988 legislation (which was updated in 1994) Con- 
gress created the Governing Board to set policy for NAEP. The 26- 
member Board is composed of policymakers, school administrators, 
teachers, curriculum and measurement specialists, business represen- 
tatives, and members of the general public. Congress charged the 
Board with specific policymaking duties that include determining 
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NAEP assessment objectives and test specifications; improving the 
form and use of NAEP; identifying appropriate achievement goals for 
each grade and subject tested; ensuring that NAEP items are free from 
bias; and selecting subjects to be assessed by NAEP. 

As stated earlier, the 1992 Writing Eramework was developed 
through a national consensus process directed by the Center for Re- 
search on Evaluation, Standards, and Student Testing. The Erame- 
work was adopted by the Governing Board in August 1990 and 
formed the basis for the 1992 NAEP in writing. Eor the 1998 Writing 
Assessment, the Governing Board contracted with ACT, Inc. to aug- 
ment the Eramework with a set of Writing Assessment and Exercise 
Specifications. The Specifications Development work occurred dur- 
ing 1995-96. 
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Chapter 1: Developing the 
Writing Framework 



The Nature of Writing and the NAEP Writing 
Assessment 

A s the National Council of Teachers of English Commission 
on Composition has noted: 

Writing is a powerful instrument of thought. In the act of 
composing, writers learn about themselves and their world and 
communicate their insights to others. Writing confers the power to 
grow personally and to effect change in the world. 

— Commission on Composition, National Council of Teachers of 
English, 1979. 

Since 1969, the National Assessment of Education Progress 
(NAEP) has conducted six assessments to measure the writing 
achievement of students in our nation’s schools. Each time, student 
writing samples were collected and scored on a national basis. The 
results of these assessments have contributed significantly to the ex- 
panding body of research on written composition and writing assess- 
ment. In each assessment and set of writing test objectives, NAEP 
attempted to reflect advances in writing instruction and measurement. 
This sixth national assessment of writing draws heavily on NAEP’s 
prior experience while adding refinements based on recent research 
and best practice. 

Three types of writing are assessed — narrative, informative, and 
persuasive. Persuasive writing focuses on exerting an impact on the 
reader. Narrative discourse emphasizes the writer’s experiences, per- 
ceptions, and imagination. Writing for informative purposes stresses 
the subject matter that is being explained. A fuller description of these 
types of writing appears in chapter 2. 
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The Writing Process 

Recent research and practice have indicated that focusing on what 
students do as writers, rather than on theory and grammar, results in 
more effective written communication. Unfortunately, instruction in 
the writing process often prescrihes a simple linear formula: from 
prewriting (generating and organizing ideas) to writing to postwriting 
(revising and editing). In reality, all three stages in the process are 
interactive and recursive. Composing involves a variety of plans and 
suhprocesses that are brought to hear as needed (Hillocks, 1986). 
Cooper and Odell (1977) define the writing process as follows: 

Composing involves exploring and mulling over a subject; plan- 
ning the particular piece... getting started; making discoveries about 
feelings, values, or ideas, even while in the process of writing a draft; 
making continuous decisions about diction, syntax, and rhetoric in 
relation to the intended meaning and to the meaning taking shape; 
reviewing what has accumulated, and anticipating and rehearsing 
what comes next; tinkering and reformulating; stopping; contemplat- 
ing the finished piece and perhaps, finally, revising. 

Good instruction helps students learn that while one or another 
phase of the writing process might be emphasized at a given time, 
other phases may come into play as well. For example, good writers 
revise at all stages, not just at the completion of their work. 

The NAEP Writing Assessment supports the process approach to 
writing in several ways. It provides substantial time for writing — not 
just the opportunity to create a rough draft. It offers suggestions for 
prewriting and revising, and includes a special prewriting page for 
each topic. 

The Development Process for the Writing 
Framework 

The Writing Framework in this booklet was developed through a 
national consensus process conducted by the Governing Board. The 
Governing Board convened a 14-member panel composed of writers, 
writing teachers, other educators, and representatives of business and 
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professional organizations. The writing framework panel met three 
times between December 1989 and March 1990. 

To gather a wide range of input, the Board also sent letters solicit- 
ing recommendations to a broadly representative group of educators, 
administrators, state and local government representatives, and mem- 
bers of the business community, the press, and the general public. The 
writing framework panel used information from the nearly 100 re- 
sponses received to inform its deliberations. Although the final 
Framework does not reflect the views of everyone who participated, it 
does represent, as nearly as possible, the consensus of the various 
groups. 

As a result of this process, the writing framework panel developed 
positions — subsequently endorsed by the Governing Board — that 
highlight elements of continuity as well as important changes in the 
1992 assessment. The main principles are as follows: 

All Students Should Write 

Learning to write well is important for all American students. Even 
though students have varied backgrounds and experiences, the expec- 
tation for high performance in writing applies to all. 

Overarching Objectives 

Writers exhibit varying degrees of competency, and the writing 
framework panel recognized that no single assessment can fully 
evaluate performance across the entire domain. The overarching ob- 
jectives presented in chapter 2 define the boundaries or focus of the 
writing assessment and ensure that it measures a variety of important 
writing skills. 

Instructional Practices 

For many years, NAEP writing assessments have interpreted 
achievement data in the context of instructional practices to which 
students have been exposed. Such information will be gathered from 
both students and their teachers on methods, materials, and opportuni- 
ties for writing instruction. Clearly, not all students receive the same 
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instruction in writing or have equal opportunities to practice or to re- 
ceive feedback on the quality of their efforts. By relating writing in- 
structional practices to student achievement, NAEP can provide a 
richer information base by which to interpret results. 

Instructional Relevance and Validity 

NAEP has long struggled with the difficulties of writing assess- 
ment. Many educators feel that the constraints of the testing situation 
seriously limit the usefulness of the test results. Recent NAEP writing 
assessments have tried to deal with these issues in two important 
ways. Eirst, the amount of time for writing tasks has been extended. 
Second, portfolio assessment has been introduced on a trial basis to 
collect samples of students’ writing in response to regular classroom 
assignments of varied complexity and duration. 
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Chapter 2: Designing the NAEP 
Writing Assessment 



Rationale and Objectives for the Assessment 

T he Writing Assessment Framework is organized according to 
three primary purposes for writing — narrative, informative, 
and persuasive. The Framework incorporates the findings of 
past NAEP writing assessments, as well as ideas from exemplary state 
frameworks and recent research on composition. 

The assessment is designed around the following six overarching 
ohjectives: 

• Students should write for a variety of purposes (narrative, infor- 
mative, and persuasive). 

• Students should write on a variety of tasks and for many different 
audiences. 

• Students should write from a variety of stimulus materials and 
within various time constraints. 

• Students should generate, draft, revise, and edit ideas and forms 
of expression in their writing. 

• Students should display effective choices in the organization of 
their writing. They should include detail to illustrate and elaborate 
their ideas and use appropriate conventions of written English. 

• Students should value writing as a communicative activity. 

The following section contains specific information on each ohjec- 
tive, including definitions, rationales, and methods of assessment. 

Objective 1: Students should write for a variety of purposes: 
narrative, informative, and persuasive. 

The NAEP Writing Assessment examines student responses to 
these three major types of writing. The purposes for writing are de- 
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rived from the interaction among the reader, the writer, and the sub- 
ject. Although other types of writing also might he defined, the writ- 
ing panel and responses from the canvassing letter concurred with 
these three hroad writing types. Because NAEP serves as a national 
monitor of student achievement — not as an individual diagnostic 
test — assessment of these hroad writing types is appropriate and con- 
sistent with NAEP’s role. 

Each type of writing is characterized hy distinguishing features and 
requires different strategies on the part of the writer. Eor example, a 
personal narrative requires decisions about the chronology of events 
and what details to include; a persuasive letter requires that the stu- 
dent focus on an issue and choose what type of appeals (e.g., emo- 
tional or logical) to direct at the reader. Although a particular piece of 
writing may have one major purpose, there may be secondary pur- 
poses as well. Purposes may blend in various ways depending on the 
specific context for writing. 

As Cooper and Odell (1977) note, “The effectiveness of a particu- 
lar sample of writing require[s] the blending of the pure colors of the 
theoretical system into the earthier shades of actual performances.” 

Narrative Writing 

Narrative writing involves the production of stories or personal es- 
says. Practice with these forms helps writers to develop an ear for 
language. Also, informative and persuasive writing can benefit from 
many of the strategies used in narrative writing. Eor example, there 
must be an effective ordering of events when relating an incident as 
part of a report. 

Sometimes narrative writing contributes to an awareness of the 
world as the writer creates, manipulates, and interprets reality. Such 
writing — whether fact or fiction, poem, play, or personal essay — 
requires close observation of people, objects, and places. Eurther, this 
type of writing fosters creativity, imagination, and speculation by al- 
lowing the writer to express thoughts and then stand back, as a more 
detached observer might, and grasp more fully what is being felt and 
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why. Thus, narrative writing offers a special opportunity to analyze 
and understand emotions and actions. 

Informative Writing 

Informative writing focuses primarily on the suhject-matter ele- 
ment in communication. This type of writing is used to share knowl- 
edge and to convey messages, instructions, and ideas. Like all writ- 
ing, informative writing may he filtered through the writer’s impres- 
sions, understanding, and feelings. Used as a means of exploration, 
informative writing helps both the writer and the reader to learn new 
ideas and to reexamine old conclusions. Informative writing may also 
involve reporting on events or experiences, or analyzing concepts and 
relationships, including developing hypotheses and generalizations. 
Any of these types of informative writing can he based on the writer’ s 
personal knowledge and experience or on information newly pre- 
sented to the writer that must be understood in order to complete a 
task. Usually, informative writing involves a mix of the familiar and 
the new, and both are clarified in the process of writing. Depending 
on the task, writing based on either personal experience or secondary 
information may span the range of thinking skills from recall to 
analysis to evaluation. 

Persuasive Writing 

Persuasive writing emphasizes the reader. Its primary aim is to in- 
fluence others to take some action or bring about change. Persuasive 
writing may contain great amounts of information — facts, details, ex- 
amples, comparisons, statistics, or anecdotes — but its main purpose is 
not simply to inform but to persuade. This type of writing involves a 
clear awareness of what arguments might most affect the audience 
being addressed. Writing persuasively also requires use of critical 
thinking skills such as analysis, inference, synthesis, and evaluation. 

Persuasive writing is called for in a variety of situations. It may in- 
volve responding to a request for advice by giving an opinion and 
providing sound reasons to support it. It may also involve presenting 
an argument in such a way that a particular audience will find con- 
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vincing. When there is opposition, persuasive writing may entail re- 
futing arguments that are contrary to the writer’s point of view. 

In all persuasive writing, authors must choose the approach they 
will use. They may, for instance, use emotional or logical appeals or 
an accommodating or demanding tone. Regardless of the situation or 
approach, persuasive writers must he concerned with having a par- 
ticular desired effect upon their readers, heyond merely adding to 
knowledge of the topic presented. 

Objective 2: Students should write on a variety of tasks and 
for many different audiences. 

Students gain power and flexibility as writers when they are given 
frequent opportunities to write for varied purposes, topics, and audi- 
ences. Awareness of the intended audience and purpose of a specific 
writing task affects the ideas that are included, the way in which they 
are organized, and the manner in which they are expressed. 

Writing tasks may ask students to draw exclusively on their own 
experience or may require use of subject matter presented in school. 

In NAEP, some writing prompts require students to base their re- 
sponses on their own ideas, knowledge, or experience; others require 
use of information provided in the prompt itself. The entire pool of 
writing topics represents a wide array of forms of writing, including 
essays, letters, stories, and reports. 

Writing to different audiences requires attention to appropriate con- 
tent and tone, depending on whether the audience is adult or the stu- 
dents’ peer, known or unknown, knowledgeable or uninformed about 
the topic, or known to be friendly or hostile. Although any formal 
writing assessment has the teacher or grader as the implied major au- 
dience, students may be asked to write for a variety of audiences in- 
cluding friends, local government officials, relatives, or business rep- 
resentatives. In NAEP, some topics ask students to write for a particu- 
lar audience (e.g., a peer, school principal, or committee). Eor other 
topics the audience is not specified. 
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Objective 3: Students should write from a variety of stimu- 
lus materials, and within different time constraints. 

In actual writing situations, the writer may be responding to a writ- 
ten stimulus, illustration, or other material. In the assessment, students 
are asked to respond to a variety of stimulus materials, in addition to 
brief written directions. For example, stimulus materials in the as- 
sessment include letters, poems, brief reports or descriptions, and 
other extended texts. The NAEP assessment also includes illustrations 
such as pictures and graphics as stimulus material for writing. 

The assessment provides students at all three grade levels with 25 
minutes to write on each of two specific tasks. Although real-world 
writing constraints may range from several minutes to many months, 
the assessment conditions are designed to allow students a reasonable 
time to respond in a thoughtful, organized manner. Enough time is 
given to develop, evaluate, and revise the written responses. The time 
limit of 15 minutes per task used in previous NAEP assessments was 
deemed insufficient to produce valid responses to the extended writ- 
ing prompts that are included. 

The NAEP writing special study allows students to submit samples 
of writing produced in the regular classroom setting. A special study 
of classroom-based writing was conducted in grades 4 and 8 in 1990 
and repeated at these grades in 1992 and 1998. 

Objective 4: Students should generate, draft, evaluate, re- 
vise, and edit ideas and forms of expression in their writing. 

Good writers develop their own writing processes based on per- 
sonal experience. The process varies among individual writers. Even 
the same writer does not always approach a writing task the same 
way. When students start to understand and control a writing task, 
they are beginning to manage their writing. The typical steps in the 
writing processes are planning, drafting, evaluating, and revising, but 
a neat linear progression is not implied. To meet high standards stu- 
dents may engage in various steps in the writing process again and 
again until they are satisfied with their work. 
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With NAEP’s increased response time, students have more oppor- 
tunity to engage in writing processes. Within the limits of a 25-minute 
time period, they should he able to generate ideas and provide infor- 
mation to support them. Students may express their ideas and organ- 
ize their response in an outline, list, word weh, or other means. With 
the time constraints in mind, students should then move to the stage 
of composition during which they draft the material in sentence and 
paragraph form. Then changes are developed through revisions, in 
which students demonstrate their ability to evaluate, revise, or edit as 
the final form evolves. The revision stage shows students’ under- 
standing of the technical aspects needed to shape their work toward a 
particular form. Finally, as time becomes an increasingly important 
factor, students should reach a decision regarding the combination of 
content and form and move toward a more finished product. At this 
stage, the writers should once again evaluate their work, as they put 
the finishing touches on what comprises the most effective response 
to the writing task. 

NAEP provides several opportunities for students to demonstrate 
their writing and report on the writing processes they use. Back- 
ground questions ask how frequently they engage in steps such as 
planning and editing when they write in school. Other questions ask 
about the use of computers and word processors. In the assessment 
itself, students are encouraged to use a special page for planning, or 
prewriting. The directions accompanying the prompts give sugges- 
tions on how to approach the writing task and improve responses 
(e.g., by planning and revising). 

Of course, a formal assessment can only go so far in mirroring 
classroom or real-life writing situations. The writing samples pro- 
duced by students in 25 minutes cannot be viewed as final or polished 
drafts. Also, it is not possible to incorporate into a timed, secure as- 
sessment such as NAEP certain writing process strategies, such as 
using outside resources and peer or teacher conferences. Changes in 
the administration should make the NAEP writing assessment more 
closely resemble process approaches to writing. 
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As noted earlier, NAEP also is experimenting with collecting class- 
room-based writing samples. These samples will enable students to 
demonstrate writing processes in which they engage over extended 
periods of time, their choice of writing topics, and their use of addi- 
tional resources. 

Objective 5: Students should display effective choices in the 
organization of their writing. They should include detail to 
illustrate and elaborate their ideas, and should use appro- 
priate conventions of written English. 

To write effectively, students must organize ideas coherently, 
elaborate their points with appropriate detail, and employ the conven- 
tions of English grammar and usage. Organization often depends on 
the task. A shopping list organized in order of importance or location 
involves specific organizational skills, but these skills are different 
from those required to develop a research paper. Regardless of the 
writing task, however, a student who writes effectively will select a 
suitable organizational form and adapt it to the task. Having a reper- 
toire of organizational strategies helps students better manage the 
various stages of writing. 

Elaboration involves the ability to select specific points or details 
and effectively incorporate them into written work. By using appro- 
priate details, a piece of writing conveys concretely the writer’ s inten- 
tions. Understanding the need for elaboration and the extent to which 
it must be carried out indicates a writer’ s ability to recognize the writ- 
ing content and to work within constraints — either those prescribed 
by the task or the intended audience, or those imposed by the writer. 

One important aspect of the writing process is the student’s ability 
to incorporate effective supportive material into a given piece of writ- 
ing. The selection of quotations, examples, anecdotes, and other 
forms of detail show the writer’s expertise in choosing material that 
enriches a given writing task. The choices a writer makes and the ex- 
planations attached to those choices, provide insight into the writer’s 
ability to synthesize ideas. Such decisions are, in part, determined by 
considerations of audience and purpose. Eor example, illustrative ma- 
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terial used in a timed, analytical essay exam will differ from that in a 
reflective piece of personal writing. The choices made show the 
writer’s ability to integrate content-hased decisions within a technical 
framework that is appropriate for the task at hand. 

Ohjective 5 is assessed in the evaluation of student responses. All 
student papers are scored using the NAEP enhanced primary trait 
method, which focuses on how well students accomplish the writing 
task. Scoring guides provide raters with specific criteria for each 
score point in terms of the appropriate content, organization, and 
elaboration for each particular writing task. The scale also incorpo- 
rates features related to use of the conventions of standard written 
English. 

Objective 6: Students value writing as a communicative 
activity. 

Both common sense and empirical data support the belief that peo- 
ple who value particular endeavors tend to invest more time and en- 
ergy in them. Writing should be a valued activity. Students should be 
engaged in their writing tasks, understand the importance of writing, 
write often, value their own efforts, and recognize good writing by 
others. Past assessments have included objectives emphasizing the 
value that students place on writing. Yet, values are always difficult 
to measure. In NAEP, students are asked directly about their opinions 
of various writing tasks and to describe their own, nonassigned writ- 
ing outside of school. Because the writing framework panel believed 
in the importance of student perceptions about writing, it encouraged 
NAEP to continue to refine its approaches to assessing such con- 
structs. 
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Chapter 3: Constructing and 
Scoring the Assessment 



Designing Topics 

T he design of NAEP writing tasks or prompts considers the fac- 
tors that affect student writing performance. The chart in fig- 
ure 1 illustrates how the pool of prompts should represent im- 
portant, appropriate, and feasible characteristics. Given financial con- 
straints, the assessment cannot include all prompts, nor are all equally 
desirable or feasible. The features in figure 1 should be used as a 
guide for item writers in the development of prompts that contain a 
reasonable distribution of the above characteristics. A brief descrip- 
tion of each feature follows. 

Discourse Aim 

As mentioned in chapter 2, in NAEP writing assessments three 
general types of writing are assessed: narrative, informative, and per- 
suasive. To accomplish writing to these different purposes, writers 
must use different strategies and content; however, each general cate- 
gory of purpose permits a variety of types of writing within it. Eor 
example, narrative writing may include story writing or narrating a 
personal experience. Informative writing may require students to ex- 
plain the meaning of an important quotation or excerpt from a speech. 
Persuasive writing may involve defending an argument or position, or 
exploring a problem and its solution. A major characteristic of any 
writing prompt is the purpose for writing and its accompanying 
dominant structure and content. 

Topic 

The subject of any writing is also one of its major features. Writing 
prompts may ask students to rely exclusively on their own back- 
ground and experiences or may ask them to use subject-matter 
knowledge learned in school or presented in the prompt itself. Many 
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writing tasks ask students to draw on school-related experiences. All 
topics are carefully screened for gender, racial, cultural, and regional 
biases. Finally, writing topics must be within the realm of experience 
for students at each particular grade level assessed by NAEP. 

Cognitive Complexity 

Each writing assignment may also be categorized according to the 
major form of reasoning required. Summaries require students to dis- 
till important information about a subject. Analyses require sorting 
information into categories and explaining each one. Interpretations 
require drawing inferences or conclusions from evidence. Evaluations 
require applying criteria of significance or worth to support a judg- 
ment or argument. Describing the major type of reasoning required by 
a writing prompt allows educators to judge whether it covers impor- 
tant types of higher order thinking. 



Figure 1 — Features To Consider in the Design of Writing 
Tasks 



Discourse Aim 
Major aim — narrative, 
informative, persuasive 
Subgenre — for example, 
position paper, story, letter 

Topic 

Information source — 
personal experience, 
school, new information 
Familiarity 
Interest 

Cognitive Complexity 
Rec all/summarize 
Analyze 
Infer/interpret 
Evaluate 



Audience 
Known/ unknown 
Adult/child 
Novice/expert 
Friendly/unfriendly 

Presentation Format 
Written 
Pictorial 

Evaluation Criteria 
Administration Conditions 
Writing Procedures Measured 
Prewriting/planning 
Drafting 
Revising 
Editing 
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Audience 

Any formal writing assessment has the teacher or scorer as the im- 
plied major audience. However, through the design of the writing 
tasks, an assessment can vary the intended audience to have the stu- 
dent address others such as peers, relatives, newspaper editors, or 
politicians. Writing to different audiences requires attention to appro- 
priate content and tone, depending on whether the audience is an 
adult or peer, known or unknown, knowledgeable or uninformed 
about the topic, and friendly or unfriendly. 

Presentation Format 

The prompt may be presented in written form only or it may be ac- 
companied by an illustration, such as a picture or graphic. NAEP in- 
cludes more prompts that ask students to write from relatively long 
passages about school subjects. Certain prompts also use illustrations 
and charts as stimulus material. 

Evaluation Criteria 

The prompt may contain directions to the students that indicate 
how their writing will be evaluated. It may present features or stan- 
dards by which the composition will be judged. Although some topics 
provide only a brief description of the writing task, more prompts 
were developed in which evaluation criteria are included. 

Administration Conditions 

The NAEP assessment includes prompts at all grade levels that al- 
low 25 minutes to respond to a particular writing task. During the as- 
sessment, students are not permitted to use books or other resources 
or to obtain advice from teachers or peers. 

Writing Procedures Measured 

The extent to which students engage in prewriting, organizing, and 
drafting will be analyzed and reported as part of the writing results. 
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Scoring the Assessment 

NAEP uses scoring criteria that include primary trait, holistic, and 
mechanics elements, and incorporate features related to task accom- 
plishment, overall fluency, and the conventions of standard written 
English. The separate NAEP trend reports will continue to provide 
results using separate holistic and mechanics scoring criteria as well. 
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Writing Assessment and 
Exercise Specifications 




Introduction 



T he Nation’s Report Card/’^ which reports data from the Na- 
tional Assessment of Educational Progress (NAEP), is the 
only nationally representative and continuing assessment of 
what America’s students know and can do in various academic suh- 
jects. Since 1969, assessments have been conducted periodically in 
reading, writing, mathematics, science, history, geography, and other 
fields. By making ohjective information on student performance 
available to policymakers at the national, state, and local levels, 

NAEP is an integral part of our nation’ s evaluation of the condition 
and progress of education. 

NAEP is a congressionally mandated project of the National Center 
for Education Statistics (NCES) in the U.S. Department of Education. 
NCES is responsible, by law, for carrying out the NAEP project. In 
1988, Congress established the National Assessment Governing 
Board to set policy for NAEP. The Board is responsible for selecting 
subject areas to be assessed, for developing assessment objectives and 
specifications through a national consensus approach, for setting ap- 
propriate student performance levels, and for performing other duties 
as required under the law. 

NAEP has conducted periodic assessments of students’ writing 
achievement over the past 30 years. The Writing Framework for the 
1992 National Assessment of Educational Progress was developed 
under the direction of the Governing Board and involved writing edu- 
cators, policymakers, scholars, teachers, and professionals in the field 
of writing. This framework, drawing on NAEP’s three decades of ex- 
perience in direct writing assessment, recommended increased time 
for writing, a wider variety of stimulus materials, suggestions and 
space to encourage prewriting, and a special writing portfolio study. 
Item development, scoring, analysis, and reporting for the 1992 as- 
sessment were conducted by Educational Testing Service, under con- 
tract to NCES. Results of the main 1992 assessment were reported in 
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the NAEP 1992 Writing Report Card, which was released in June 
1994. 



In preparation for the NAEP Writing Assessment, the Governing 
Board issued a request for proposals for the development of writing 
assessment and exercise specifications, which would augment the 
1992 Eramework. The Board awarded the contract to ACT in Sep- 
tember 1995. During the project, ACT technical and writing staff 
drew on the expertise of two committees: a Planning Committee of 
teachers, writers, scholars, researchers, and curriculum coordinators; 
and a Technical Committee of psychometricians and state testing di- 
rectors. The resulting specifications for the NAEP Writing Assess- 
ment are intended to guide item development and test construction, 
and to produce an assessment on which NAEP achievement levels 
can he set. Hundreds of individuals and organizations participated in 
reviewing drafts of the specifications document. The Governing 
Board unanimously adopted the specifications at its May 1996 
meeting. 
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Overview of the NAEP Writing 
Assessment 



T he National Assessment of Educational Progress (NAEP) Writ- 
ing Assessment will evaluate students’ abilities to use their 
individual writing processes and appropriate writing strategies 
to compose with clarity, ease, and precision. Assessment exercises 
will provide the opportunity to engage in a variety of writing proc- 
esses. Important aspects of writing include the act of invention 
through different pre writing strategies, the ability to write for varied 
purposes and audiences, the knowledge and use of various revision 
strategies, and attention to correctness through editing and proofread- 
ing skills. 

The assessment was designed to be consistent with the overarching 
objectives identified in the Writing Eramework for the 1992 National 
Assessment of Educational Progress. These five major assessment 
objectives are: 

• Students should write for a variety of purposes: narrative, infor- 
mative, and persuasive. 

• Students should write on a variety of tasks and for many different 
audiences. 

• Students should write from a variety of stimulus materials and 
within various time constraints. 

• Students should generate, draft, revise, and edit ideas and forms 
of expression in their writing. 

• Students should display effective choices in the organization of 
their writing. They should include detail to illustrate and elaborate 
their ideas, and use appropriate conventions of written English. 

The assessment calls for a variety of written responses to tasks 
within the confines of a large-scale writing assessment. The written 
responses will be viewed and evaluated as first draft, not polished, 
writing. The limitations of any large-scale writing assessment do not 
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allow for a complete revision and refinement process. The NAEP 
Writing Assessment is offered as a single assessment that cannot fully 
evaluate performance across the entire domain of writing. However, 
the results do provide valuable information about student ability to 
generate first-draft writing in response to a variety of purposes, tasks, 
and audiences. The following pages present detailed specifications for 
the overall assessment and for the tasks. 

A Prompt Writer’s Guide setting forth basic rules for good con- 
struction is to be provided by the assessment development contractor. 
The Guide will include criteria for developing tasks using a combina- 
tion of stimulus materials and prompts and must conform to the speci- 
fications set forth in this document and the “National Assessment 
Governing Board Policy on Cognitive Item Development and Re- 
view,” as well as any formatting requirements of NAEP. 

The following specifications for the assessment are divided into 
two sections: Assessment Specifications and Task Specifications. The 
Assessment Specifications section provides an overall description of 
the construction, review, and scoring of the assessment and defines 
how the assessment should be built. The Task Specifications section 
describes the construction of the assessment in terms of content and 
format. 

As indicated in the Writing Framework for the 1992 National As- 
sessment of Educational Progress, the Writing Assessment will 
measure three purposes, or modes, of writing: narrative, informative, 
and persuasive (see table 1). The Eramework document proposed 
these three broad types of writing to assist in the organization of the 
domain of writing, although it is recognized that other modes of writ- 
ing are possible and an overlap of modes is probable. 

The Purposes of Writing 

The purposes of writing are offered as one way of describing the 
domain and are not intended to place constraints on the students’ writ- 
ing. The selection of these three modes of writing was based on the 
importance of the modes as commonly found in instruction. This dis- 
tinction of mode, although recognized as artificial, is often a helpful 
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distinction for novice writers. In addition, the modes are offered as a 
means to ensure that the NAEP Writing Assessment covers a wide 
range of tasks. Because NAEP serves as a national monitor of student 
achievement, assessment of these hroad writing types was seen as 
appropriate and consistent with NAEP’s role. These three hroad writ- 
ing types are defined in the Eramework as: 

Narrative 

Narrative writing involves the production of stories or personal es- 
says. Practice with these forms helps writers to develop an ear for 
language. Also, informative and persuasive writing can benefit from 
many of the strategies used in narrative writing. Eor example, there 
must he an effective ordering of events when relating an incident as 
part of a report. Sometimes narrative writing contributes to an aware- 
ness of the world as the writer creates, manipulates, and interprets 
reality. Such writing — whether fact or fiction, poem, play, or personal 
essay — requires close observation of people, objects, and places. Eur- 
ther, this type of writing fosters creativity, imagination, and specula- 
tion by allowing the writer to express thoughts and then stand back, as 
a more detached observer might, and grasp more fully what is being 
felt and why. Thus, narrative writing offers a special opportunity to 
analyze and understand emotions and actions. 

Informative 

Informative writing focuses primarily on the subject-matter ele- 
ment in communication. This type of writing is used to share knowl- 
edge and to convey messages, instructions, and ideas. Eike all writ- 
ing, informative writing may be filtered through the writer’s impres- 
sions, understanding, and feelings. Used as a means of exploration, 
informative writing helps both the writer and the reader to learn new 
ideas and to reexamine old conclusions. Informative writing involves 
reporting on events or experiences, or analyzing concepts and rela- 
tionships, including developing hypotheses and generalizations. Any 
of these types of informative writing can be based on the writer’ s per- 
sonal knowledge and experience or on information newly presented to 
the writer that must be understood in order to complete a task. Usu- 
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ally, informative writing involves a mix of the familiar and the new, 
and both are clarified in the process of writing. Depending on the 
task, writing based on either personal experience or secondary infor- 
mation may span the range of thinking skills from recall to analysis to 
evaluation. 

Persuasive 

Persuasive writing emphasizes the reader. Its primary aim is to in- 
fluence others to take some action or to bring about change. Persua- 
sive writing may contain great amounts of information — facts, details, 
examples, comparisons, statistics, or anecdotes — but its main purpose 
is not simply to inform but to persuade. This type of writing involves 
a clear awareness of what arguments might most affect the audience 
being addressed. Writing persuasively also requires use of critical 
thinking skills such as analysis, inference, synthesis, and evaluation. 

Persuasive writing is called for in a variety of situations. It may in- 
volve responding to a request for advice by giving an opinion and 
providing sound reasons to support it. It may also involve presenting 
an argument in such a way that a particular audience will find con- 
vincing. When there is opposition, persuasive writing may entail re- 
futing arguments that are contrary to the writer’s point of view. 

In all persuasive writing, authors must choose the approach they 
will use. They may, for instance, use emotional or logical appeals or 
an accommodating or demanding tone. Regardless of the situation or 
approach, persuasive writers must be concerned with having a par- 
ticular desired effect upon their readers, beyond merely adding to 
knowledge of the topic presented. 

The NAEP Writing Assessment will be developed in these three 
modes at each grade level but will not be equally divided at each 
grade level (see table 2). Although students are capable of writing in 
all three modes at all grade levels, the modes receive different empha- 
ses in instruction at each grade level. The assessment will support the 
process approach to writing without requiring a specific approach of 
students, as some students may be unfamiliar with any particular ap- 
proach. The assessment will provide substantial time for writing, will 
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offer suggestions for prewriting and drafting when appropriate, and 
will provide an opportunity for prewriting exercises within the test 
booklet. 

Developmental Changes in Students’ Understand- 
ing of Writing Processes 

Developing student writers are expected to achieve an increasingly 
hroad and deep knowledge and understanding of the value of writing 
in their lives, of their own individual writing processes, of the range 
of writing strategies available to them, and of the benefits of sharing 
and publishing their writing for a wider audience. The following dis- 
cussion seeks to show how these developmental changes manifest 
themselves at grade levels 4, 8, and 12 with the assumption that stu- 
dents have participated in a well-developed instructional writing pro- 
gram. The assessment development committee should draw upon this 
discussion to ensure that the exercises they are constructing are age- 
appropriate. 

Grade 4 

By the fourth grade, students should have the critical skills, vo- 
cabulary, and concepts that allow them to use school and leisure time 
to write. Personal choices for writing, fluency of ideas, and freedom 
of expression are emphasized. They are developing an understanding 
that there are many stages to the writing process, including prewrit- 
ing/brainstorming, composing/drafting, revising, editing/ 
proofreading, and sharing/publishing. They understand that each writ- 
ing task does not necessarily entail all stages of the writing process. 
Fourth-grade students have a growing awareness of their own indi- 
vidual writing processes and the personal choices open to them. 

Fourth-grade students write for public and private purposes in a va- 
riety of literary forms, including poems, stories, reports, and personal 
narratives. They write to persuade, using order of importance and 
classifying differences and likenesses (or advantages and disadvan- 
tages). They use writing according to purpose and intended audience. 
They compose individually and collaboratively. Their developing 
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awareness of revision strategies involves a move from the deliberate, 
systematic, and concrete to a tentative, flexible, risk-taking, large- 
scale revision process. Fourth-grade students are becoming aware of 
many alternatives, of new possibilities, through the writing process. 
Fourth-grade students gather information and ideas from a variety of 
sources, including personal experiences and literature. They add in- 
formation and ideas to early drafts in developing writing projects. 
They write across the curriculum for formal and informal purposes, in 
various modes of discourse, and for a variety of audiences, including 
themselves. 

Fourth-grade students respond to the writing of peers in pairs and 
small groups. They demonstrate a sense of authorship by sharing and 
publishing writing. They are learning to critically view their own and 
others’ work. They revise for specific and precise language and for 
sequencing of paragraphs. They develop editing and proofreading 
skills, which include editing for word choice and expanding basic 
sentence patterns. They proofread — individually and collabora- 
tively — for conventional usage, spelling, capitalization, and punctua- 
tion. They apply appropriate conventions for dialogue and quotation. 
They demonstrate the use of conventions for different documents, 
such as letters and reports. 

Grade 8 

In addition to the knowledge and skills developed at the fourth- 
grade level, eighth-grade students recognize and use author tech- 
niques, such as appeals to reason and emotion, figurative language, 
and satire, which demonstrate a sense of audience when composing 
their own texts. Eighth-grade students have a deeper understanding of 
the stages of the writing process and are developing a wider range of 
writing strategies. 

Eighth-grade students have a growing awareness of their own indi- 
vidual writing processes and the range of personal choices they may 
make. 

Eighth-grade students write in an expanding repertoire of forms, 
which includes letters to the editor, directions, and summaries. 
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Eighth-grade students are able to explain and demonstrate how writ- 
ten communication is affected hy choices writers make in language, 
tone, and voice, and why some choices are considered more appropri- 
ate than others. These students have a developing sense of personal 
voice that varies with purpose and audience. 

Eighth-grade students will reflect on their own developing literacy, 
set goals, and evaluate their progress. Eighth-grade students respond 
to various written, visual, and electronic texts, and make personal 
connections between these texts and their own lives. Eighth-grade 
students use writing mechanics that clarify meaning. 

Grade 12 

At the 12th-grade level, students have extended capabilities in the 
use of written forms that include satire, drama, interview, precis, 
evaluation, and analysis. Twelfth-grade students have an enhanced 
understanding of the stages in the writing process and a recognition 
that all writing tasks need not go through each stage in the process. 
Twelfth-grade students have deep insight into their own writing proc- 
esses and the varied writing choices open to them. 

Twelfth-grade students recognize and use innovative techniques 
such as stream of consciousness and multiple viewpoints to convey 
meaning and engage an audience. These students recognize and dem- 
onstrate ways in which communication can be manipulated through 
word usage as in propaganda, sarcasm, and humor. Twelfth-grade 
students have an enhanced sense of personal voice that is demon- 
strated for different audiences and purposes. Twelfth-grade students 
will use various strategies when constructing meaning in writing and 
will develop strategies to deal with new communication needs. 

Twelfth-grade students write analytical responses to various 
texts — written, visual, and electronic — making personal connections 
with their own lives. Twelfth-grade students reflect on their under- 
standing of literacy, set personal learning goals, create strategies for 
attaining those goals, and take responsibility for their literacy devel- 
opment. Twelfth-grade students will identify and manipulate writing 
mechanics that enhance and facilitate understanding. 
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Section I: Assessment 
Specifications 



Content Specifications 

T he Planning Committee recommended the development and 
inclusion a variety of tasks within the three modes of writing. 

A task is defined as the combination of stimulus materials and 
accompanying prompts. A prompt, which usually presents a rhetorical 
situation, topic, purpose, and audience, is that part of the task that re- 
quests a student’s response. The details of the content specifications 
are provided helow. 

• Increase the size of the task pool. The number of tasks developed 
as part of the 1992 assessment was far too low. The size of the 
task pool should be increased in an effort to increase the degree of 
freedom in selection and administration at the operational stage. 

A minimum of 20 tasks will be included in the Writing Assess- 
ment for each grade level. 

• Develop a variety of stimulus materials. The developed task pool 
will provide a variety of stimulus materials and accompanying 
prompts to reflect the large number of tasks that are typically 
asked of students. The newly developed pool will be large enough 
to cut across a wide range of real-life experiences. Although the 
writing prompts will draw from real-life experiences, caution 
should be exercised in the interpretation of the resulting writing 
samples. It must be clear that a large-scale writing assessment can 
go only so far to mirror real-life writing experiences. The writing 
samples produced will not be viewed or evaluated as final or pol- 
ished drafts. 

• Provide extensive prewriting and revising guidelines. Students 
use varied approaches to the prewriting process, such as brain- 
storming, mapping, and free writing; however, not all students 
employ a formal process approach to writing. The Planning 
Committee recommended that some structure be provided within 
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the testing experience. Students will be provided directions for 
scaffolding exercises such as thinking, planning, or drafting. They 
also will be provided with a checklist of questions that helps them 
to focus on a particular task. The administration of each task will 
be separately timed and will allow the students an opportunity to 
work through topic generation, selection, and development. The 
opportunity for student engagement in these activities will not re- 
quire separately timed activities within the testing block. Rather, 
students will be able to structure their time within the block to ac- 
commodate their own writing strategies. Although the Planning 
Committee recognized the importance that collaborative work 
plays in the writing classroom, it is not logistically possible to in- 
corporate such strategies as peer or teacher collaboration in a 
timed, secure assessment such as NAEP. 

• Design the scoring rubrics to be grade specific. The scoring ru- 
brics will be designed to be grade specific. These within-grade 
rubrics will be customized for each task that is developed. The fi- 
nal scoring rubrics will be determined after the field test papers 
have been evaluated. As presented in the Task Specifications sec- 
tion to follow, scoring rubrics will include criteria on content, or- 
ganization, form, language, and control of mechanics (spelling, 
punctuation, grammar, and capitalization). 

Technical Specifications 

The Technical Committee recognized the importance of the content 
specifications and strongly recommended that the content specifica- 
tions be the first priority of the test developers. However, in addition 
to content specifications, an iterative development process is being 
recommended where the test developers and psychometricians will 
work collaboratively with content specialists to develop a test that is 
both content valid and technically sound. 

Precise recommendations for distributions, intercorrelations, and 
levels of difficulty are not contained in this document; however, ap- 
propriate ranges are provided as guides for the test developers. Based 
on results of the last field test administration, these values seem 
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achievable given that the size of the task pool is being increased to 
150 tasks. Decisions about distributions of task responses, task diffi- 
culty and discrimination, and intertask correlations will be made as a 
collaborative effort between the test development contractor’ s techni- 
cal and content staff after the field test results have been scored and 
analyzed. 

Recommendations for the technical specifications are: 

• Adhere strictly to the content specifications. 

• Consider content specifications as your primary responsibility. 
However, assuming adherence to the content specifications, field 
test an adequate sample of tasks (75) to allow for the identifica- 
tion and selection of the best possible tasks. 

• Given the design of the writing field test, select tasks within the 
pool of field-tested tasks that demonstrate a reasonable level of 
intertask correlation. Selected tasks should not be so unique that 
generalizability is limited. 

• Encourage appropriate analyses and data explorations following 
the field test administration. Given that the size of the field test 
administration will not support some types of analyses, use ob- 
servable statistics such as means, variances, and frequency distri- 
butions to help select operational prompts. 

• Seek a spread of difficulty (i.e., the average score for a particular 
prompt) within a moderate range (with averages or means ranging 
from 2.0 to 4.0, assuming a 6-point scale). The means should be 
uniformly distributed within that range. This is not to infer that all 
tasks should be similar in format or approach. Rather, given a va- 
riety of tasks, the overall performance on the tasks selected for 
inclusion in the operational test should be in the moderate range 
of difficulty. 

• Assuming a 6-point scale, seek a task standard deviation of .90 or 
greater to help ensure a spread of responses. Seeking a standard 
deviation of at least .90 and mean difficulties in the range of 2.0 
to 4.0 will help to ensure that student responses will be spread 
across all possible score points. 
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• Consider the balance between task variety and the generalizability 
of the tasks. For example, consistent with the Planning Commit- 
tee’s recommendations, the tasks will offer a wide variety of real- 
life experiences. However, the tasks will not be so unique from 
one another that they limit the ability to generalize beyond the 
particular prompt. 

• Consider the impact of the interaction between score -point distri- 
butions and interrater agreement (the percent of time that one 
reader’s assigned scores agree with a second reader’s assigned 
scores). High agreement statistics between readers may indicate 
the use of few score points. High agreement statistics should be 
examined in addition to interrater correlations to ensure both vari- 
ability across the scoring rubric and consistency of assigned 
scores. 

• Seek tasks that broadly discriminate over the entire scoring ru- 
bric. Tasks shall elicit responses at all possible score points. As- 
suming a 6-point scale, at least two percent of the respondents 
shall be found in each of the extreme values (1 and 6). Based on 
the field test results, revise or eliminate tasks that do not elicit re- 
sponses at all score points, assuming adherence to the content 
specifications. 

• Counterbalance the administration of the tasks to control for con- 
text and position effects. 

Review Specifications 

Writing Expert Review 

To ensure the development of tasks that adequately represent the 
content domain and exhibit proper psychometric characteristics, as 
well as to construct a task pool that will adequately measure the proc- 
esses, skills, and knowledge described in the three achievement lev- 
els, it is important that review by writing educators and practicing 
writers be incorporated at several points during the assessment devel- 
opment process. Therefore, the development, field testing, and selec- 
tion of tasks will be monitored by an assessment development panel. 



44 




A minimum of 20 percent of the membership of the Planning Com- 
mittee will serve on this panel as specified by National Assessment 
Governing Board policy. 

After the tasks have been developed, the panel will review the pool 
at each grade level and judge the tasks for congruence with the speci- 
fications document. The tasks will be judged on criteria such as 
grade-level appropriateness, technical accuracy, content validity, va- 
riety of formats, and the mode of writing that references the tasks to 
the assessment dimensions they purport to measure. In addition, re- 
viewers will ensure that the available pool is balanced, that it is repre- 
sentative of the content described in the achievement level defini- 
tions, and that it incorporates sufficient tasks in each writing mode at 
the various achievement levels for each grade. 

Tasks will be reviewed again after field test administration, as part 
of the process of selecting those that will appear in the operational 
assessment. Any tasks that statistical evaluation reveals to be techni- 
cally flawed will be eliminated. 

Bias Review 

All tasks will be screened for evidence of cultural bias and for lack 
of ethnic and gender sensitivity, and will be subjected to postfield test 
analyses. The field test administration samples will be selected to be 
as representative of the NAEP operational sample as possible. 

Given the lack of evidence supporting the use of Differential Item 
Functioning statistics for tasks of this type, the Technical Committee 
recommended the examination of frequency distributions for consis- 
tency of response patterns for the various groups of interest. If, after 
close scrutiny, an exercise appears to be a valid measure of writing, 
and if no plausible explanation for a differential performance is ap- 
parent, the task will be retained. As mandated by law, the National 
Assessment Governing Board has final review authority of all cogni- 
tive items prior to field testing and, subsequently, prior to the opera- 
tional assessment. 
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Administration Specifications 

Preparation 

The Planning Committee recommended that the students he told 
what is expected of them prior to the testing experience. In an effort 
to give students every opportunity to write as well as they can, an ah- 
hreviated version of the scoring rubric will he distributed prior to the 
testing experience. This version may be a checklist of the criteria that 
will be ultimately used in scoring. Facilitators will use the abbreviated 
scoring rubrics to enhance students’ understanding of how their writ- 
ing samples are scored. 

After the test, students will be provided questions that would help 
to evaluate their opportunity to learn particular types of writing. 

Training of Facilitators 

Extensive training will be provided to ensure that facilitators are 
appropriately prepared to administer the writing assessment. The 
training materials should cover topics such as room arrangements for 
students, preparation for testing, administration procedures, and 
guidelines for the amount and type of interaction that can be allowed 
between examinees and facilitators. 

Ethical Considerations 

The writing assessment tasks will need to be sensitive to the pri- 
vacy of students and not ask them to reveal sensitive personal experi- 
ences. Further, the tasks will not ask students about privileged home 
information or psychological information. 

Scoring and Reporting Specifications 

The Planning Committee recommended that within-grade scoring 
rubrics be developed. These within-grade rubrics will be customized 
for each task that is developed. The papers selected as anchor papers 
will be specific to the grade level and to the task. The final scoring 
rubrics will be determined after the field test papers have been evalu- 
ated. However, the general characteristics of writing to include in the 
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scoring rubrics for each of the three different modes of writing are 
contained in appendix A. For all three modes, the student’s demon- 
stration of control of mechanics (spelling, punctuation, capitalization, 
grammar) is one characteristic of writing that will be included in the 
overall rubric. 

Previous experience with the NAEP Writing Assessment has 
shown that a 6-point score range was often reduced to 4 or 5 points, 
since very few papers received the highest scores. To help avoid this 
problem, the development of grade-appropriate scoring rubrics is rec- 
ommended. The scoring contractor will work with the test develop- 
ment contractor to select papers at each point on the score scale for 
each task. It will be critical to obtain multiple and varied examples of 
student writing at each possible score point for each task. If such ex- 
amples cannot be located, the task will be considered for revision or 
elimination from the pool of available tasks. These selected examples 
will serve to further define and elaborate on the language of the rubric 
and will be used to train the readers. 

The Planning Committee drafted preliminary descriptions of the 
three NAEP achievement levels for writing. The committee defined 
what constitutes Basic, Proficient, and Advanced levels of achieve- 
ment in writing at each grade level in terms of student writing. These 
preliminary descriptions encompass the characteristics of writing as 
defined by the modes of writing and as described in the guidelines for 
scoring rubrics. These descriptions will be used as input into the 
achievement level-setting process and will be refined at that time. 
Detailed descriptions of the preliminary achievement levels can be 
found in appendix B. 

In general, the following recommendations for scoring were made: 

• Einalize the characteristics of writing as a function of the re- 
sponses obtained during the field test administration. 

• If technically feasible, provide an overall or composite writing 
score that aggregates across modes of writing and different types 
of tasks. 
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• Develop within-grade scoring rubrics that are specific to a par- 
ticular task. These rubrics will include mechanics (spelling, punc- 
tuation, capitalization, grammar) as one component of the overall 
rubric. 

• Assemble training materials specific to each task. 

• Develop scoring rubrics to prevent restriction of student perform- 
ance as much as possible. For example, at the “6” level, define a 6 
paper not as perfection, but as one that could contain minor flaws; 
communicate clearly to readers that 6 contains a wide range of 
papers from “true 6s” to exemplary papers. Make sure rubrics are 
grade appropriate — that is, 4th-graders should be able to receive a 
6. Ensure that anchor papers are selected to illustrate the range of 
all score points, especially the lowest range of the 6s. 

The results of the NAEP Writing Assessment will specify the type 
of writing tasks that were asked of participating students. Reported 
results should emphasize that the NAEP Writing Assessment is a 
measure of students’ ability to draft a response to a provided task 
completed under a timed administration. Given the limitations of the 
test administration and sampling design, reported results should cau- 
tion against overinterpretation. Important differences between class- 
room writing and the NAEP Writing Assessment should be consid- 
ered as the results are released. Eor example, the availability of time 
and opportunity for writing about a chosen topic, peer or teacher re- 
sponse, self-assessment, and revision are opportunities available to 
the classroom teacher that are not afforded to a national writing as- 
sessment being administered under standardized conditions. Class- 
room-based writing assessments involve a process that begins with 
time for pre writing and first-draft writing on a student’s topic of 
choice, multiple audiences for feedback, self-assessment of strengths 
and areas needing improvement, and thoughtful revision on content 
and mechanics. A national standardized writing assessment such as 
NAEP works within the constraints of large-scale writing assessment, 
topic choice and peer or teacher response are not possible, and the 
time for self-assessment and revision is shortened. The reported re- 
sults should reflect these realities and recognize that the NAEP Writ- 
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ing Assessment does not measure students’ abilities to produce a pol- 
ished piece of writing. 

Scaling Issues 

The Technical Committee concurred with the Planning Commit- 
tee’s recommendation to combine the scales across modes of writing, 
if feasible. The Technical Committee recommended that the dimen- 
sionality of the data be investigated and used to empirically establish 
whether scaling can be done across modes of writing. 

Given the difficulty in predicting statistical characteristics of tasks 
(difficulty, discrimination, intertask correlations), the task pool will 
be increased in an effort to provide more degrees of freedom in selec- 
tion and administration at the operational stage. The committees ac- 
knowledged that the number of tasks that were developed as part of 
the 1992 assessment were far too few. 

All results reported about the NAEP Writing Assessment will em- 
phasize that the NAEP Writing Assessment is representative of stu- 
dents’ ability to write within the constraints of a large-scale writing 
assessment administration. Participants in the achievement-levels set- 
ting process will be instructed to treat the responses as examples of 
first-draft writing. The committees made the following recommenda- 
tions: 

• The spread of student scores across the entire scoring rubric shall 
be maximized to help ensure the setting of achievement levels. 

• The tasks will be developed to elicit a range of performance. 

Reporting by Subgroup Performance 

When reporting results for each grade level and for demographic 
subgroups, procedures should be used that are consistent with current 
NAEP practice. 
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Section II: Task Specifications 



T he NAEP tasks will be developed in conjunction with panels 
of task writers who are actively engaged in language arts in- 
struction as well as professional writers. Task writers will be 
instructed to draft tasks that maintain a broad focus to allow for 
choice of response for a given task, variations in classroom style, and 
other concerns specific to a student or teacher within the context of a 
given task. 

Specific to the persuasive task development, the Planning Commit- 
tee recommended that students be provided the opportunity to per- 
suade within the realm of their experiences. The text of the task 
should not be too prescriptive. Including persuasive writing at grade 4 
will provide a baseline for persuasion that could be compared across 
years. 

Writing tasks in each of the three modes will be administered to 
small, local samples prior to the field test administration. The format 
of the task, the time allocation, the spread of the responses, and the 
accessibility of the tasks will be evaluated and used to refine the tasks 
in preparation for the field test administration. All tasks will then be 
administered to nationally representative samples of students to help 
determine the appropriateness of each task for operational use. The 
test administration sessions will be carefully monitored to determine 
the types of problems that students encounter with each task. Re- 
sponses to the testing activities will be elicited from both administra- 
tors and students and used to refine the testing process. 

Student responses from the field test administration will be scored 
by readers trained by the scoring contractor. These readers have com- 
pleted an undergraduate degree in writing, education, or a related 
field. The readers will be asked to describe the success of each task in 
terms of how well it elicits a range of student writing and in terms of 
their reaction to reading the papers. Readers’ responses will be care- 
fully considered and the prompts that are reviewed unfavorably will 
be eliminated. Given the difficulty with imposing “mode” into the test 
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specifications, readers will be aware of the possibility of students re- 
sponding to particular prompts using a variety of modes. Readers 
will be trained how to deal with situations such as this in the scoring 
process. 

Based on information from the field test (e.g., frequency distribu- 
tions of students at each of the score levels and means and standard 
deviations of scores for each prompt), 25-minute tasks will be 
grouped into test blocks for the operational administration. The ap- 
propriateness of the tasks within a particular block will be based on 
both content and technical considerations. The final versions of the 
test forms will be subjected to several reviews to ensure that the 
forms conform to good test construction practice. As in the develop- 
ment stages, these reviews will include both content and sensitivity 
consultants from across the nation. 

Format Specifications 

The Planning Committee recognized the importance of motivation 
and its interaction with student performance. The committee recom- 
mended providing a variety of stimulus materials, varying the presen- 
tation of the test, providing manipulative materials, and providing 
visual presentations of the prompt. The tasks will be made as interest- 
ing and user-friendly as possible. Table 1 reflects the types of tasks 
that will be developed. 

The test booklet will invite planning and revision, but not require it. 
However, some structure within the testing experience will be pro- 
vided for the students to assist with the prewriting and planning proc- 
esses. Recognizing that there are a variety of ways in which to ap- 
proach the process of writing, no specific approach to prewriting will 
be imposed on students. Rather, students will be provided directions 
for scaffolding activities such as thinking, planning, and drafting for 
the appropriate tasks. They may also be provided with a checklist of 
questions that help them to focus on a particular task. Examples of 
such scaffolding activities are provided in appendix C. 

Given the different approaches to writing by grade level, the Plan- 
ning Committee recommended that the prompts be developed to be 
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grade specific. For example, fourth-grade persuasive prompts would 
be specifically developed as opposed to general persuasive prompts. 
Key to developing successful prompts within each grade level will be 
identifying the appropriate context for students. All prompts need to 
invite the desired type of writing as determined by one of the three 
modes. However, they also need to invite writing that is appropriate 
to the grade level of the students. Providing the student with stimulus 
materials similar to those typically found in classroom assignments at 
grades 4, 8, and 12 is one way to select appropriate prompts. The 
prompts also must allow for a range of responses, be clear and ex- 
plicit in their instructions to the student, and be stated in language 
appropriate to the grade level of the student. 



Table 1 — Illustrative Examples of Writing Tasks 





Grade 4 


Grade 8 


Grade 12 


Narrative 


Provide visual 
stimuli of a season 
of the year. Ask 
students to write a 
letter to a grand- 
parent telling the 
story of an interest- 
ing personal ex- 
perience related to 
the season. 


Provide visual 
stimuli. Ask stu- 
dents to write an 
article for a sports 
magazine telling 
the story of a time 
when they par- 
ticipated in a 
hobby or skill 
they enjoyed. 


Provide an appro- 
priate quotation. 
Ask students to 
write a letter to a 
friend telling the 
story of a time in 
their lives when 
they had to make 
an important deci- 
sion. 


Informative 


Provide an appro- 
priate quotation. 
Ask students to 
explain in an essay 
to their English 
teacher how a per- 
son (parent, 
teacher, friend) has 
influenced them in 
an important way. 


Provide a series 
of brief journal 
entries from an- 
other historical 
time. Ask stu- 
dents to explain 
what is revealed 
about the person 
who wrote the 
entries. 


Provide quota- 
tions from a po- 
litical campaign. 
Ask students to 
choose one and in 
an essay inform 
their social stud- 
ies teacher what it 
means in the con- 
text of the cam- 
paign. 
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Grade 4 


Grade 8 


Grade 12 


Persuasive 


Provide visual 
stimuli of an ani- 
mal. Ask students 
to convince their 
parents/guardians 
of an animal that 
would make the 
best pet. 


Provide brief re- 
views, as models, 
of a film, TV pro- 
gram, or book. 
Ask students to 
write a review for 
the school news- 
paper that will 
convince other 
students to watch 
a favorite film or 
TV program or 
read a favorite 
book. 


Provide a quota- 
tion on education 
in the United 
States. Ask stu- 
dents to write a 
letter to the editor 
of their local 
newspaper taking 
a position on 
some aspect of 
education and 
support it from 
their own experi- 
ences. 



The size of the task pool was increased in an effort to increase the 
degrees of freedom in selection and administration at the operational 
stage. The Planning Committee acknowledged that the number of 
tasks that were developed as part of the 1992 assessment were far too 
few. A minimum of 20 tasks will he included in the Writing Assess- 
ment for each grade level. Table 2 provides a breakdown of the per- 
centage of time students in each grade shall spend on tasks that reflect 
the various modes of writing. 



Table 2 — Percentage of Time To Be Spent on Tasks for 
Each Writing Purpose (Mode) 





Grade 4 


Grade 8 


Grade 12 


Narrative 


40% 


33% 


25% 


Informative 


35% 


33% 


35% 


Persuasive 


25% 


33% 


40% 



Scoring Rubric Specifications 

Refinement of the scoring criteria will be an integral and ongoing 
part of the test development process. Advice from Planning Commit- 
tee members involved in the project was solicited in order to develop 
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criteria consistent with good writing assessment. Preliminary guide- 
lines for drafting rubrics and the general characteristics of the scoring 
rubrics are included in appendix B of this document. 

The scoring guides shall be supported by samples illustrating the 
range of variation within each score level. The writing samples in- 
cluded in the scoring guides will illustrate the range of responses that 
are selected by experienced readers who apply the scoring guide to a 
random selection of the papers for a particular task. It is critical that 
the samples selected for illustration demonstrate low-, moderate-, and 
high-performance levels at each possible score point. For example, on 
a 6-point scale, examples of low 5s, middle 5s, and high 5s will be 
contained in the training materials. This is particularly critical for 
score points at the extreme ends of the scale (1 and 6). These samples 
will then become the basis of the training materials that are used dur- 
ing the scoring process. 

Reader Training Specifications 

Readers will be trained to read and score a piece of writing by 
evaluating it against the following factors: 

• A score point description specifically designed for a particular 
writing assessment. 

• Anchor papers chosen through a consensus process. 

Readers will be provided with extensive training before beginning 
to score student writing samples. Training staff will explain to readers 
the goals of the assessment and its relationship to the scoring process. 
These goals will be reiterated throughout the training, keeping an em- 
phasis on the students whose writing is being evaluated. 

Training will provide readers with the opportunity to read a se- 
lected sample of student papers and understand the broad range of 
writing performance demonstrated by the students. After discussing 
the sample papers, readers will have the opportunity to practice scor- 
ing additional samples of papers. 

Prior to scoring writing samples for a particular form or mode, each 
reader will be required to qualify by scoring at least two sets of papers 
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that have been previously scored hy expert readers. The scores as- 
signed hy each reader to the qualifying sets will he compared to those 
assigned hy the expert readers. 

Reader performance will he monitored throughout the scoring 
process, both statistically and by random second reading by an expert 
reader. Readers will be provided with statistics showing how their 
scores compare to fellow readers with whom they have been paired, 
allowing them and their supervisors to check for any tendency toward 
inaccurate or inconsistent scoring. Reader reliability will be deter- 
mined from first- and second-reader correlations and agreement sta- 
tistics, not from adjudicated scores. 

Procedures will be established to monitor the amount of reader drift 
that occurs during the scoring process. This is typically done through 
the use of validity papers that are randomly assigned to readers 
throughout the scoring process. The relationship between reader per- 
formance and the “master” score assigned to the validity papers will 
be carefully monitored. 

To facilitate the achievement-levels setting process, all papers used 
for the achievement-levels setting process will be double scored. The 
performance of readers on validity papers will be checked, and any 
necessary recalibration of readers will be done prior to the scoring of 
papers to be used in the achievement-levels setting process. 
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Appendix A 



Achievement Level Descriptions for Grade 4 
Writing 

Achievement Level Descriptions for Grade 8 
Writing 

Achievement Level Descriptions for Grade 12 
Writing 




Achievement Level Descriptions for Grade 4 
Writing 

These achievements levels are proposed for first drafts, not final or 
polished student writing, that are generated within limited time con- 
straints in large-scale assessment environment. 

Basic 

Fourth-grade students performing at the Basic level should he able to 
produce a somewhat organized response within the time allowed that 
shows a general grasp of the writing task they have been assigned. Their 
writing should include some supporting details. Its grammar, spelling, and 
capitalization should be accurate enough to communicate to a reader, al- 
though there may be mistakes that get in the way of meaning. 

Proficient 

Fourth-grade students performing at the Proficient level should be able to 
produce an organized response within the time allowed that shows an un- 
derstanding of the writing task they have been assigned. Their writing 
should include details that support and develop the main idea of the 
piece, and its form, content, and language should show that these stu- 
dents are aware of the audience they are expected to address. The 
grammar, spelling, and capitalization in the work should be accurate 
enough to communicate to a reader; there may be some mistakes, but 
these should not get in the way of meaning. 

Advanced 

Fourth-grade students performing at the Advanced level should be able 
to produce an effective, well-developed response within the time al- 
lowed that shows a clear understanding of the writing task they have 
been assigned. Their writing should be clearly organized, making use 
of techniques such as consistency in topic or theme, sequencing, and a 
clearly marked beginning and ending. It should make use of precise and 
varied language to speak to the audience the students are expected to 
address, and it should include details and elaboration that support and 
develop the main idea of the piece. Their writing may also show signs of 
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analytical, evaluative, or creative thinking. The grammar, spelling, and 
capitalization in the work should he accurate enough to communicate 
clearly; mistakes should he so few and so minor that a reader can eas- 
ily skim over them. 



Achievement Level Descriptions for Grade 8 
Writing 

These achievement levels are proposed for first drafts, not final or 
polished student writing, that are generated within limited time con- 
straints in a large-scale assessment environment. 

Basic 

Eighth-grade students performing at the Basic level should he able 
to produce an effective response within the time allowed that shows a 
general understanding of the writing task they have been assigned. Their 
writing should show that these students are aware of the audience they are 
expected to address, and it should include supporting details in an organ- 
ized way. The grammar, spelling, punctuation, and capitalization in the 
work should be accurate enough to communicate to a reader, although 
there may be mistakes that get in the way of meaning. 

Proficient 

Eighth-grade students performing at the Proficient level should be 
able to produce an effective response within the time allowed that 
shows an understanding of both the writing task they have been as- 
signed and the audience they are expected to address. Their writing 
should be organized, making use of techniques such as sequencing or a 
clearly marked beginning and ending, and it should make use of details 
and some elaboration to support and develop the main idea of the piece. 
Their writing should include precise language and some variety in sen- 
tence structure, and it may show analytical, evaluative, or creative think- 
ing. The grammar, spelling, punctuation, and capitalization in the work 
should be accurate enough to communicate to a reader; there may be 
some errors, but these should not get in the way of meaning. 
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Advanced 



Eighth-grade students performing at the Advanced level should he 
able to produce an effective and fully developed response within the 
time allowed that shows a clear understanding of both the writing task 
they have been assigned and the audience they are expected to 
address. Their writing should show some analytical, evaluative, or 
creative thinking, and should demonstrate precise word choice and 
varied sentence structure. Their work should include details and 
elaboration that support and develop the main idea of the piece, and it 
may make use of strategies such as analogies, illustrations, examples, 
anecdotes, or figurative language to clarify a point. At the same time, 
the writing should show that these students can keep their work 
clearly and consistently organized. Writing by eighth-grade students 
performing at the Advanced level should contain few errors in 
grammar, spelling, punctuation, capitalization, and sentence structure. 
These writers should demonstrate good control of these elements and 
may use them for stylistic effect in their work. 

Achievement Level Descriptions for Grade 12 
Writing 

These achievement levels are proposed for first drafts, not final or 
polished student writing, that are generated within limited time con- 
straints in a large-scale assessment environment. 

Basic 

Twelfth-grade students performing at the Basic level should be able 
to produce an effective response within the time allowed that shows 
an understanding of both the writing task they have been assigned 
and the audience they are expected to address. Their writing should 
show some analytical, evaluative, or creative thinking. It should include 
details that support and develop the central idea of the piece, and it 
should be clearly organized, making use of techniques such as a consis- 
tency in topic or theme, sequencing, and a clear introduction and conclu- 
sion. The grammar, spelling, punctuation, and capitalization in these stu- 
dents’ work should be accurate enough to communicate to a reader; 
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there may be some errors, but these should not get in the way of 
meaning. 

Proficient 

Twelfth-grade students performing at the Proficient level should be 
able to produce an effective and fully developed response within the 
time allowed that uses analytical, evaluative, or creative thinking. 

Their writing should be organized effectively, and it should show that 
these students have a clear understanding of the writing task they 
have been assigned. It should be coherent, making use of techniques 
such as a consistent theme, sequencing, and a clear introduction and 
conclusion, and it should include details and elaboration that support 
and develop the main idea of the piece. The writing should show that 
these students are able to use precise language and variety in sentence 
structure to engage the audience they are expected to address. Writing 
by 12th-grade students performing at the Proficient level should contain 
few errors in grammar, spelling, punctuation, capitalization, and sen- 
tence structure. These writers should demonstrate a command of these 
elements and may use them for stylistic effect in their work. 

Advanced 

Twelfth-grade students performing at the Advanced level should be 
able to produce a mature and sophisticated response within the time 
allowed that uses analytical, evaluative, or creative thinking. Their 
writing should be fully developed, incorporating details and elabora- 
tion that support and extend the main idea of the piece. It should show 
that these students can use literary strategies — anecdotes and repeti- 
tion, for example — to develop their ideas. At the same time, the writ- 
ing should be well crafted, organized, and coherent, and it should in- 
corporate techniques such as consistency in topic or theme, sequenc- 
ing, and a clear introduction and conclusion. It should show that these 
writers can engage the audience they are expected to address through 
rich and compelling language, precise word choice, and variety in 
sentence structure. Writing by 12th-grade students performing at the 
Advanced level should contain few errors in grammar, spelling, punc- 
tuation, capitalization, and sentence structure. These writers should 
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demonstrate a sophisticated command of these elements and may use 
them for stylistic effect in their work. 
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Appendix B 



A Suggested Process for Rubric Construction 
General Characteristics of Writing by Mode 




A Suggested Process for Rubric Construction 

1 . Convene a group of writing experts and classroom teachers to 
discuss the nature of the assessment (e.g., number and nature of 
tasks, time allowed). 

2. Read a wide sampling of field-test papers, looking for special 
characteristics of the students contained in the sample as these 
characteristics will influence the level of complexity in the infor- 
mation specified hy the ruhric. Also, look through the student re- 
sponses to get an idea of the diversity of responses and levels of 
achievement to identify the characteristics and content that should 
he included in the ruhric. 

3. Consider the level of discriminations necessary for the purpose of 
the test. Consider the length of time the student has had available 
to respond to the task. 

4. Read all the papers and divide them into piles that demonstrate 
the characteristics of writing that are described in the rubric for 
each score point. 

5. Write descriptors for each pile of papers. Consider what charac- 
teristics distinguish the top papers from the lowest levels. Then, 
assess what categories these characteristics fall into. In assessing 
writing, for example, the categories of most rubrics fall into pur- 
pose, audience, idea development/support, organization/structure, 
sentence structure, and word choice, voice, and mechanics. 

6. Write rough drafts of descriptors for each score point. 

7. Consider the rubric to be a “draft in process” until after the field 
test results have been evaluated. 

General Characteristics of Writing by Mode 

Narrative 

• Understands the narrative purpose. 

• Develops character. 
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• Maintains focus. 

• Has satisfying resolution. 

• Has appropriate ordering of events. 

• Gives attention to audience when appropriate to the prompt. 

• Uses elaboration and details. 

• Handles direct and indirect discourse. 

• Demonstrates control of mechanics. 

Informative 

• Understands the informative purpose. 

• Has clear and complete information. 

• Conveys messages, instructions, and/or ideas. 

• Uses sufficient detail. 

• Uses coherent and logical organization. 

• Shows efficient relationships between and among ideas. 

• Gives attention to audience. 

• Fulfills the demands of the task. 

• Uses language level appropriate to the topic and voice desired by 
the writing. 

• Demonstrates control of mechanics. 

Persuasive 

• Understands the persuasive purpose. 

• Takes and retains a position. 

• Supports and develops a position through examples, details, sta- 
tistics, and supporting evidence. 

• Has coherent and logical organization. 

• Gives attention to audience. 
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• Uses language level appropriate to the topie and voiee desired by 
the writing. 

• Demonstrates eontrol of mechanics. 
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